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OUTLINE 

Optimum sensitivity is defined as the derivative of the optimum design with 
respect to some problem parameter, P. The problem parameter is usually fixed 
during optimization, but may be changed later. Thus, we can use optimum sen- 
sitivity to estimate the effect of changes in loads, materials or constraint 
bounds on the design without expensive re-optimization. 

Here, we will discuss the general topic of optimum sensitivity, identify 
available methods, give examples, and identify the difficulties encountered in 
calculating this information in nonlinear constrained optimization. 
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THE NEED FOR OPTIMUM SENSITIVITY 
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r* 'Xj 
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In many situations, we not only want to find the optimum, but we also want to 
know how sensitive the optimum is relative to a certain parameter (i.e. how 
stable the optimum is). 

When parameter P changes, optimum sensitivity can be used to estimate the 
changes in the optimum design variables and objective function without expensive 
re-optimization. 

In multi-level optimization, we need the derivative of the lower level op- 
timum with respect to the upper level design variables. 


1. FIND THE CHANGE IN THE OPTIMUM DESIGN DUE TO CHANGES IN LOADS, 
MATERIALS, OR OTHER DESIGN SPECIFICATIONS 

2. AVOID RE-OPTIMIZATION 

3. PROVIDE NEEDED INFORMATION FOR MULTI-LEVEL OPTIMIZATION 
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THE DEFINITION OF OPTIMUM SENSITIVITY 


The mathematical definition of optimum sensitivity is given here. What makes 
this unique from what we usually define as sensitivity analysis is that there is 
an implied inequality constrained sub-problem. Because of this, it is possible 
that the optimum sensitivity may not be continuous at P = P®. 


OPTIMUM SENSITIVITY 

DF* /DP = LIMIT [F ( X*+AX* , P+AP ) - F(X*,P)]/AP 
Ap-»0 


DX* /DP = LIMIT [AX*/AP] 
Ap-»0 


WHERE F (X*+AX , P+AP ) IS FOUND FROM; 
MINIMIZE F (X, P+AP) 

SUBJECT TO; 

G j(X,P+AP) < 0 J=1,M 

L U 

X I < X I < Xj 1=1, N 
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AVAILABLE METHODS 


Several methods have been proposed to estimate the optimum sensitivity of a 
design with respect to parameter P. Each of these methods contains certain 
assumptions, and these assumptions can be incorrect in some cases. The methods to 
be discussed here are listed below. 


1. BASED ON THE KUHN-TUCKER NECESSARY CONDITIONS FOR AN OPTIMUM 

2. BASED ON THE CONCEPT OF A FEASIBLE DIRECTION 

3. BASED ON A LINEAR PROGRAMMING METHOD 

4. BASED ON A FULL SECOND-ORDER APPROXIMATION 
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METHOD 1 


The assumption contained in this method is that all of the constraints that 
are critical at the optimum will remain critical when P changes infinitesimally. 

Differentiation of the Kuhn-Tucker conditions gives n equations. 

The assumption gives another K equation, where K is the number of critical 
constraints at the optimum. 

This method requires second-order information. 

Because of the assumption that all critical constraints remain critical, this 
method does not recognize the discontinuity which may exist in the optimum sen- 
sitivity. 

This method gives no assurance that the answer obtained is correct. 


METHOD 1; BASED ON THE KUHN-TUCKER CONDITIONS 

AT X* Gj(X*) =0 j e k 

VF (X*) + J~ XtVG j(X*) = 0 
J£K 

THIS LEADS TO THE SOLUTION OF THE FOLLOWING SET OF EQUATIONS; 


A NxN 

B NxK 


* S' 

r + i 

r °Nxl 
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WHERE 

9 2 F (X* ) V 3 2 Gj(X*) 

a IK = + ) 

3Xj3x k l 9Xj9X k 


WITH SIMILAR EXPRESSIONS FOR B IK , Cj AND Dj 


23 


METHOD 2 


This method treats the parameter as a new design variable. This enlarges the 
design space to n+1. The assumption contained in this method is that, in the ex- 
panded design space, the maximum improvement or minimum degradation in the design 
is sought. 

This method seeks the constrained steepest descent direction in n+1 space to 
give DX*/DP, and from this DF /DP is calculated directly. 

This method requires only first-order sensitivity information. 

This method accounts for possible discontinuity of the total derivative. 

As with the first method, there is no assurance that the result obtained is 
correct. 


LINEAR METHOD BASED ON FEASIBLE DIRECTIONS 

LINEAR APPROXIMATION: LET X N+1 = P 

MINIMIZE F (X) = F(X*) +VF(X*)*£ 

SUBJECT TO; 

Gj(X*) + ZPjQL*) ' 1 < 0 J £ K 

S BOUNDED 
* 

WHERE Sj = Xj - Xj IS EQUIVALENT TO Sj = 9X-J-/3P 

EQUIVALENT PROBLEM: 

MINIMIZE VF(X*)‘j> 

SUBJECT TO; 

VGj(X*)‘.S<0 J EK 
S*S < 1 
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METHOD 2B 


Two extended forms are available to deal with the possible discontinuity of 
the optimum sensitivity. This is necessary because the direction in which P is 
changed will determine the value of the sensitivity. If the value is different, 
depending on the sign of delta-P, then this indicates that the design will follow 
one subset of constraints if P is increased but a different set if P is decreased. 

This method for dealing with the potential discontinuities of the optimum 
sensitivity is somewhat dependent on the choice of the parameter C. Numerical 
difficulties can be encountered in deciding the correct value of C. 


DEALING WITH DISCONTINUITY DEPENDENT ON THE SIGN OF P 
AP > 0 

MINIMIZE VF(X*)’S^ - C’S N+1 
SUBJECT TO; 

VGj(X*)‘S < 0 J EK 
S*S < 1 

AP < 0 

MINIMIZE VF (X*) *j5 + C * S N+1 


SUBJECT TO; 


METHOD 3 


Here, we create a Taylor series expansion for the objective function and the 
critical constraints. Taking the limit as delta-P goes to zero and keeping the 
lowest order terms only produces the optimum sensitivity according to the 
original definition. This process requires that we pay close attention to whether 
delta-P approaches zero from the positive or negative side. 

This method requires solving two resultant LP problems. 

This method requires first-order sensitivities only. 

If the number of the constraints is less than the number of design variables, 
the LP problems do not have a unique solution. 

If a unique solution exists, it is always the correct solution. 


LINEAR PROGRAMMING APPROACH 
USING THE DEFINITION OF OPTIMUM SENSITIVITY; 

MINIMIZE VF(X*)‘AX + 3F(X*)/3P‘AP + 0(AX,AP) 

SUBJECT TO; 

Cj(x\ p ) + VGj(X*)'_M + 9Gj/8P‘ AP + 0(AX,AP) < 0 J=1,M 

KEEP THE LOWEST ORDER TERMS WHEN Ap-»0. THIS LEADS TO; 

IF AP -> +0 (AP>0): 

MINIMIZE VF(X*) , AX/DP + 3F(X*)/9P 

SUBJECT TO; 

VGj(X*)\AX/DP + 3Gj/3P <0 J EK 

IF AP -> -0 (AP< 0): 

minimize vf(x*)* ax/dp - 3f(x*)/sp 

SUBJECT TO; 

VGj(X*)*Ax/DP - 3Gj/9P <0 J^K 
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METHOD 4 


Just as with method 1, this method requires second derivatives. However, 
here the second-order information is used directly as an approximate optimization 
task. 


The parameter P may be treated as an independent design variable, or the 
change in P may be specified. 

If a small change in P is specified, the method becomes a finite difference 
method. When delta— P goes to zero, this method gives the exact answer to a second 
order approximation. 

The set, K, of retained constraints can include all critical and near criti- 
cal constraints, or even the entire set of constraints. Therefore, as P is 
changed, a totally new set of constraints can become critical. 

Within the limits of numerical precision, this method will always give the 
correct solution. The disadvantage is that this problem has a quadratic objective 
and constraints and so must be solved by nonlinear programming. It is, however, 
quite efficient since it is an explicit problem. 

If an attempt is made to simplify this method by linearizing it, the result 
is the set of two LP problems given in method 3. 


FULL SECOND-ORDER APPROXIMATION 

SOLVE THE FOLLOWING EXPLICIT APPROXIMATE PROBLEM: 
FIND THE CHANGE £ THAT WILL 

MINIMIZE F (X*,P) + VF(X*,P)-S^ + O.SsjHpS^ 
SUBJECT TO; 

g jQL*> p > + Z. G jQL*> p ) ML + 0.5s t HjS <o j e k 
S BOUNDED 


27 




AN OPTIMUM CURVE 


When P changes, the optimum points X* form a curve in n+1 space. The optimum 
sensitivity DX /DP is represented by the tangent of this curve. The curve can be 
nonsmooth, so DX /DP can be discontinuous. 

An infinitesimal change in P may cause the curve to leave a currently criti- 
cal constraint. This demonstrates the potentially discontinuous nature of optimum 
sensitivity. 


P 



DISCONTINUOUS DERIVATIVES 


This is another graphical example of the discontinuous derivative problem. 

In this case, the constrained optimum is found for P=0 to lie on the constraint 
boundary. Now if P is increased, the optimum sensitivity will point to the uncon- 
strained minimum. On the other hand, if P is decreased, the optimum sensitivity 
follows the constraint. Since the total derivative is the scalar product of the 
gradient of the objective function^with the vector S, it is clear that the optimum 
sensitivity is not continuous at X . 


P 
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ROTATING SHAFT OPTIMIZATION 


This example demonstrates the usefulness of optimum sensitivity as an en- 
gineering approach to frequency domain avoidance. Assuming it is required that 
the rotating shaft not vibrate in the domain between 2.8 and 3.5 Hz, the shaft is 
first optimized with respect to all other constraints. Then the sensitivity with 
respect to the fundamental frequency is calculated and a new optimum design is 
projected with a frequency below 2.8 Hz and with a frequency above 3.5 Hz. From 
this it appears that it is far more economical to drive the frequency up than to 
drive it down. However, this was not known in advance and so it was not known 
whether the frequency should be bounded from above or below. Thus, optimum sen- 
sitivity provides one means of dealing with a problem in which the design space is 
disjoint. 



OBJECTIVE: MINIMUM WEIGHT. CONSTRAINTS: STRESS, DISPLACEMENT, EULER BUCKLING, 

SHELL BUCKLING. PARAMETER P: THE FIRST NATURAL FREQUENCY. 

THE OPTIMUM WITHOUT ANY FREQUENCY CONSTRAINTS: ^ = 3.1, W* = 27,242 



OPTIMUM SENSITIVITY 

METHOD 2 METHOD 3 

RE -OPTIMIZE 

<2.8 

W* 

+5,278 

+5,278 

+6,429 


X 1 

-1.65 

-1.65 

CM 

• 

i — i 
1 


x 2 

40. 33 

+0.33 

+0.44 

W 1 > 3 ’ 5 

W* 

+417 

+417 

+169 


X 1 

+0.17 

+0.17 

40.17 


X2 

-0.003 

-0.003 

-0.011 
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10-BAR TRUSS 


Here the common 10-bar truss was optimized and the sensitivity was calculated 
with respect to the allowable stress in member 9. It is known that the weight of 
this structure can be reduced by increasing this allowable stress to a value of 
37.5 ksi, but beyond that, no weight reduction is possible. At the initial 
optimum, member 10 was at its lower bound. Method 1 assumed, incorrectly, that it 
would stay there, while method 2 recognized that this member dimension should be 
increased. Using method 3, the allowable stress in member 9 was allowed to change 
as an independent variable and this method projected that the optimum allowable 
stress is 38.2 ksi, quite close to the actual value of 37.5 ksi. The case at the 
bottom of the figure is for optimization at the 37.5 ksi value and shows the dis- 
continuity of the optimum sensitivity. 



OBJECTIVE: MINIMUM WEIGHT. CONSTRAINTS: STRESS, MINIMUM GAGE. 

PARAMETER P: STRESS LIMIT IN MEMBER 9. INITIAL OPTIMUM Og = 30 KSI, W* = 1545 

SENSITIVITY 


CASE 1: 

PARAMETER 

METHOD 1 

METHOD 2 

METHOD 3 


df*/dc 9 

-240.5 

-238.4 

-178.6 


S 10 

0.00 

0.17 

0.16 


a 9 

— 

— 

38.2 

CASE 2: a 9 

=37.5 KSI, 

W* = 1498, 

METHOD 2. 


Aa 9 0 

DF*/D a 9 = 215.7 W*(35) = 1512 

CALCULATED W*(35) 

Aa 9 o 

df*/d° 9 = 0.00 
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CONCLUSIONS 


Optimum sensitivity in linear programming is a common and widely used tool. 
Research in optimum sensitivity for nonlinear problems has not been this success- 
ful and it has been shown here that none of the methods is completely 
satisfactory. Methods 1-3 often do not provide the correct answer, while method 4 
requires second-order information that may be costly to obtain, as well as the 
nonlinear optimization of the approximating functions. 

The reasons for these difficulties are now beginning to be understood. If 
the optimum design is fully constrained and unique, the optimum sensitivity can be 
reliably calculated, just as in linear programming. However, if the design is not 
fully constrained (fewer active constraints than design variables), the optimum 
sensitivity using first-order information will not be unique and second-order in- 
formation is essential. Unfortunately, this is the usual case in engineering 
design. The reason that first-order information is inadequate is that the higher 
order terms cannot be ignored as delta-P goes to zero in the limit. 

The need to calculate the optimum sensitivity is a clear one and often jus- 
tifies considerable effort. It is this information that is needed to make many 
fundamental design decisions. Therefore, improved understanding of these concepts 
is useful in the search to extract the maximum information from the optimization 
process . 


1. IN GENERAL, THERE ARE SITUATIONS WHERE NONE OF THE AVAILABLE METHODS EXCEPT 
THE FULL SECOND-ORDER APPROXIMATION WILL GIVE THE CORRECT ANSWER 

2. ITERATIVE METHODS USING FIRST- AND SECOND-ORDER INFORMATION SHOULD BE 
INVESTIGATED 

3. IF SECOND-ORDER INFORMATION IS AVAILABLE, METHOD 4 WILL PROVIDE USEFUL 
ENGINEERING WHICH ACCOUNTS FOR NEARBY CONSTRAINTS THAT MAY BECOME CRITICAL 
WHEN PARAMETER P IS CHANGED 

4. FURTHER RESEARCH IS NEEDED; THE USEFULNESS OF OPTIMUM SENSITIVITY HAS BEEN 
CLEARLY SHOWN IN PAST WORK 
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